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ABSTRACT 

Current  information  systems  are  difficult  to  change  to  produce  information  that  is  tailored 
to  the  specific  needs  and  context  of  end  users.  The  information  they  produce  is  static, 
and  application  reengineering  can  be  complex,  costly  and  time  consuming,  potentially 
leading  to  system  downtime.  In  addition,  multiple  information  systems  and  sources  can 
produce  duplicate  or  inconsistent  information  that  requires  significant  human  effort  to 
correlate,  integrate  and  understand.  The  Joint  Battlespace  Infosphere  (JBI)  is  defined  as 
“a  combat  information  management  system  that  provides  individual  users  with  the 
specific  information  required  for  their  functional  responsibilities”  and  “provides  uniform 
rules  for  publishing  new  and  updated  objects  into  the  JBI  and  promptly  alerts  any  JBI 
clients  that  have  subscribed  to  such  objects.”1  The  transform  core  service  of  the  JBI 
enhances  the  value  of  information  disseminated  by  the  JBI  through  information 
manipulation  mechanisms  (fuselets)  that  tailor  the  information  space  to  the  specific  needs 
of  the  warfighter  and  mission.  This  paper  describes  the  results  of  an  experiment  that  was 
designed  to  measure  the  validity  and  value  of  the  JBI  fuselet  concept  within  the  context 
an  Air  Operations  Center  (AOC)  and  a  dynamic  collaborative  mission  replanning 
scenario. 

1.  INTRODUCTION 

1. 1  Problem  Statement 

The  information  produced  by  current  information  systems  is  typically  static  in  nature. 
These  systems  are  difficult  to  change  to  produce  information  that  is  tailored  to  the 
specific  needs  and  context  of  end  users  since  application  reengineering  can  be  complex, 
costly  and  time  consuming,  potentially  leading  to  system  downtime.  Exasperating  the 
problem  is  the  fact  that  multiple  information  systems  and  sources  can  produce  duplicate 
or  inconsistent  information  that  requires  significant  human  effort  to  correlate,  integrate 
and  understand.  This  can  lead  to  information  overload  and  confusion,  inefficient  and 
ineffective  decision-making,  and  cumbersome  and  error-prone  migration  of  information 
from  one  system  to  another,  often  requiring  manual  data  reentry. 

1.2  Joint  Battlespace  Infosphere  (JBI) 

The  JBI  is  a  vision  of  an  orchestrated  information  management  environment  whose 
services  adapt  to  the  operational  needs  of  joint  and  coalition  enterprises  for  universal 
real-time  access  to  tailorable,  actionable  information.  The  JBI  employs  publish, 
subscribe,  query,  transform,  and  control  core  services  to  deliver  decision-quality 
information  in  a  secure  and  assured  fashion  with  the  desired  Quality  of  Service  (QoS)  to 
all  users  at  all  echelons.  An  instance  of  the  JBI  is  a  dynamic  system  that  is  “stood  up”  for 
a  specific  purpose  or  mission,  and  is  scalable  and  flexible  to  the  evolving  needs  over  time 
of  a  diverse  and  changing  membership  set  of  clients  (information  producers  and 
consumers). 
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1.3  JBI  Fuse  lets  for  Information  Manipulation 

A  fuselet  is  a  special-purpose  JBI  client  program  that  provides  value-added  information 
processing  functions  that  are  under  the  control  of  the  JBI.  The  information  processing 
functions  are  crafted  to  take  existing  information  objects  as  input  and  manipulate  them  in 
some  way  to  produce  new  value-added  information  objects.  Operationally  speaking, 
fuselets  enable  information  to  be  manipulated  into  the  form  that  is  required  by  and  useful 
to  the  warfighter. 

Specifically,  the  objective  of  fuselet  technology  is  to  augment  information  systems  with  a 
flexible  information  production  capability  that  is  dynamic  to  the  changing  needs  of  end 
users  without  requiring  any  significant  changes  to  legacy  systems.  The  desired 
operational  impact  is  to  improve  the  efficiency  and  effectiveness  of  decision-making  by 
correlating  duplicative  information,  resolving  inconsistent  information,  mediating 
between  information  sources,  and  fusing  information  together  into  comprehensible 
information  products.  By  leveraging  the  managed  information  space  provided  by  the  JBI, 
flexible  and  easy  to  build  decision  logic  functions  in  the  form  of  software  components 
(fuselets)  can  be  designed  to  tailor  information  for  the  particular  purposes  of  individuals 
and  Communities  of  Interest  (COIs)1 2 3 4  so  as  to  improve  the  speed  and  effectiveness  of 
command  decisions  and  subsequent  actions. 

The  remainder  of  this  paper  describes  an  experiment  designed  to  measure  the  validity  and 
value  of  the  JBI  fuselet  concept  within  the  context  an  Air  Operations  Center  (AOC)  and  a 
dynamic  collaborative  mission  replanning  scenario.  The  name  of  the  experiment  is 
Decision-support  Infosphere  Services  for  Collaborative  Operations  and  Virtual 
Environment  Requirements  (DISCOVER). 

2.  DISCOVER:  A  Fuselet  Concept  Validation  Experiment 

2.1  Overview 

The  DISCOVER  project  is  an  experiment  designed  to  measure  the  validity  and  value  of 
the  JBI  fuselet  technology  by  applying  it  to  a  dynamic  replanning  scenario  within  an 
AOC.  In  an  attempt  to  do  so,  we  recognized  the  importance  of  attempting  to  adhere  to 
the  principles  of  the  scientific  method: 

1.  Observation  -  Dynamic  replanning  of  missions  within  an  AOC  involves  a  great 
deal  of  collaboration  (person-to-person,  person-to-device,  and  device-to-device) 
and  decision  making. 

2.  Question  -  What  information  and  information  sharing  capabilities  are  needed  to 
improve  the  effectiveness  of  decision  making  in  this  environment? 

3.  Hypothesis  -  JBI  fuselet  technology  can  produce  better  decision-quality 
information  that  will  be  integral  to  collaborative  processes  and  workflow. 

4.  Prediction  -  Fuselet  and  collaboration  capabilities  can  unobtrusively  augment 
current  planning  systems  and  make  significant  positive  improvements  to 
operational  effectiveness. 
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5.  Controlled  Experiment  -  Compare  “as-is”  and  “to-be”  operational  processes,  the 
later  of  which  incorporates  the  aforementioned  information  management 
technologies. 

6.  Theory/Learn  -  Verify  or  contradict  the  hypothesis  (or  elements  thereof)  based  on 
results  of  the  experiment. 

2.2  System  Architecture  and  Fuselet  Companion  Clients 

A  conscious  decision  was  made  to  separate  the  runtime  concerns  of  information 
production  by  a  fuselet  and  how  that  information  is  ultimately  used  or  presented,  and  by 
what/to  whom.  The  runtime  responsibility  of  a  fuselet  ends  with  the  publication  of  the 
information  it  was  intended  to  produce.  In  this  way,  all  clients  interested  in  the 
information  produced  by  a  fuselet  can  subscribe  to  it  and  do  what  they  wish  with  it 
(provided  these  clients  are  authorized  to  do  so).  However,  the  concern  over  how  the 
output  of  a  fuselet  will  initially  be  used  is  nearly  always  considered  during  the 
development  of  that  fuselet.  Recognizing  this,  there  is  the  notion  of  a  fuselet  companion 
client  that  is  designed  and  developed  in  conjunction  with  a  fuselet  (or  set  of  fuselets). 
Companion  clients  subscribe  to  fuselet  outputs  and  display  the  information  to  an  end 
user,  or  serve  as  a  bridge  (adaptor)  between  the  JBI  and  a  legacy  application  so  that 
fuselets  can  operate  on  legacy  data.  The  DISCOVER  project  involved  the  development 
of  just  such  companion  clients  which  are  components  in  the  overall  system  architecture 
illustrated  in  Figure  1. 


DISCOVER 
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Each  of  these  companion  clients  within  the  DISCOVER  system  architecture  is  briefly 
described  in  turn. 

2.2.1  JBI  Database  Bridge 

The  Theater  Battle  Management  Core  Systems  (TBMCS)  is  a  collection  of  tools,  utilities 
and  over  50  major  applications  that  have  been  integrated  to  support  Joint  Air  Operations. 
TBMCS  provides  automated  command  and  control  (C2)  and  decision  support  tools  to 
improve  the  planning,  preparation,  and  execution  of  joint  air  combat  capabilities.3  From 
the  perspective  of  the  DISCOVER  experiment,  TBMCS  is  viewed  as  a  legacy  application 
that  plays  a  part  in  the  operational  scenario  that  the  experiment  employs.  The  Air 
Operations  Database  (AODB)  and  Modernized  Intelligence  Database  (MIDB)  are 
relational  databases  used  by  TBMCS  applications.  In  order  to  demonstrate  the  capability 
of  fuselets  to  produce  information  of  operational  value  to  AOC  operators,  data  produced 
by  the  TBMCS  legacy  applications  had  to  somehow  get  published  to  the  JBI  so  that 
fuselets  could  operate  on  that  data.  The  JBI  Database  Bridge,  developed  previously  at 
AFRL,  is  used  as  a  companion  client  to  publish  TBMCS  data  into  the  information  space 
of  the  JBI  in  support  of  the  DISCOVER  experiment’s  dynamic  mission  replanning 
scenario. 

2.2.2  Unit  Status  Technician  Client 

The  Unit  Status  Technician  (UST)  companion  client  was  developed  under  the 
DISCOVER  project  allows  augmentation  of  TBMCS  data  with  information  about  the 
qualifications  of  military  units  to  handle  various  munitions.  While  this  information  is 
available  at  the  unit  level,  it  is  not  currently  available  in  the  TBMCS  force  level 
applications  within  an  AOC.  The  significance  of  this  client  is  that  unit  qualification 
information  can  be  published  to  the  JBI  along  with  the  legacy  TBMCS  data  published  by 
the  JBI  Database  Bridge.  Fuselets  then  consume  this  information  in  support  of  the 
experiment’s  operational  scenario. 

2.2.3  Collaboration  and  Visualization  Portal  Client 

In  addition  to  the  information  produced  by  fuselets,  the  DISCOVER  project  recognized 
the  requirement  to  visualize  (present)  this  information  to  human  operators  within  an  AOC 
and  the  ability  for  teams  of  operators  to  collaborate  over  this  information.  The 
DISCOVER  project  utilizes  portal,  messaging,  and  chat  technology  to  support 
presentation  and  collaboration  requirements.  Fuselets  perform  information  manipulation 
functions  on  data  stored  in  the  JBI’s  managed  information  space  in  order  to  produce 
value  added  information  that  can  be  visualized  within  a  Portal  Client.  Figure  2  shows  a 
notional  screen  shot  of  the  Portal  Client,  a  principal  DISCOVER  fuselet  companion 
client.  This  client  provides  the  primary  human-machine  interface  to  AOC  operators  in 
support  of  DISCOVER’ s  dynamic  mission  replanning  process.  The  Portal  Client 
provides  information  visualization  spaces  that  are  shared  by  multiple  AOC  operators,  a 
shared  space  for  entering  special  requirements  for  a  target,  and  Chat  and  Messaging 
services. 


4 


Figure  2  -Portal  Client  Screen  Shots 


2.3  Experimentation  Methodology 

2.3.1  Capture  and  Analyze  an  “As-ls”  Operational  Scenario  &  Process 

The  first  thing  we  did  was  consult  with  a  Subject  Matter  Expert  (SME)  who  was  both  a 
retired  Air  Force  pilot  and  AOC  operator  to  identify  an  operational  domain  where 
information  management  technology  could  be  applied  with  a  potentially  significant  return 
on  investment.  After  some  dialog,  the  SME  suggested  the  area  of  dynamic  mission 
replanning  within  an  AOC,  since  that  is  an  area  ripe  with  collaborative  and  rapid 
decision-making  activities  that  are  less  sufficiently  supported  by  existing  C2  systems  than 
are  normal  mission  planning  activities.  It  seemed  that  fuselets  might  go  a  long  way  to 
execute  the  decision  logic  to  derive  information  that  human  operators  now  have  to  find 
manually  using  current  C2  systems,  such  as  TBMCS  (see  Figure  3). 
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Figure  3  -  AOC  Collaborative  Decision-Making  Support 

Once  convinced  that  dynamic  mission  replanning  within  an  AOC  was  a  good  candidate 
for  our  experiment,  we  (in  conjunction  with  the  SME)  wrote  a  textual  description  of  the 
scenario  that  started  with  the  identification  of  a  new  high  priority  target  for  which  no 
missions  were  planned  against.  In  the  scenario,  this  eventually  led  to  the  requirement  for 
an  F-15E  Fighter  Duty  Officer  (FIDO)  to  choose  and  re-roll  a  mission  of  lower  priority  to 
attack  the  new  target.  We  were  surprised  how  complex  such  a  seemingly  straight 
forward  activity  actually  is  -  warfighting  is  by  no  means  a  paltry  affair. 

We  then  transformed  the  text-based  scenario  into  a  well-defined  process  model  (Figure  4 
captures  a  small  portion  of  the  as-is  process  model  that  we  defined).  We  used  Software 
Business  Success’s  Processworks  process  modeling  tools  and  problem  solving 
methodology  to  develop  the  model.  In  Processworks,  a  problem  (scenario)  is 
transformed  into  a  process  -  that  is,  the  problem  is  decomposed  into  a  series  of  activities 
(bubbles)  and  tasks  (text  in  bubbles),  which  are  then  related  together  by  lines  and  arrows 
designating  data  (information)  and  control  flow  throughout  the  process.  In  Processworks, 
the  specification  of  actors  relates  roles  to  responsibilities  (process  activities  and  tasks), 
and  interfaces  to  systems  that  are  responsible  for  the  production  and  storage  of 
information  produced  and  consumed  in  the  process  are  also  defined.  The  reason  an  as-is 
process  was  developed  in  Processworks  is,  first  and  foremost,  to  provide  the  basis  for 
reasoning  about  how  the  existing  process  works  and  how  it  can  be  improved. 
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Figure  4  -  Dynamic  Collaborative  Mission  Replanning  Scenario 


2.3.2  Analyze  “As-ls”  Process  for  Informational  Deficiencies 

We  next  went  into  an  analysis  of  the  process  we  defined  and  the  C2  systems  that  AOC 
operators  interact  with.  As  can  be  seen  in  Figure  4,  there  are  many  lines  of 
communication  between  people  and  systems.  Today,  much  of  the  person-to-person 
communication  is  done  by  “tennis  shoe”  interfaces,  which  makes  distributed  AOC 
operations  difficult  and  centralized  operations  vulnerable  to  attack.  Therefore, 
automation  for  collaborative  capabilities  provided  by  chat,  instant  messaging,  and  portal 
technology  would  in  theory  be  beneficial  to  speeding  up  communication  between 
operators  and  enable  it  to  be  done  in  a  distributed  way. 

Another  one  of  the  problems  that  we  identified  is  that  TBMCS  is  largely  a  sophisticated 
database  query-response  system.  For  example,  a  FIDO  could  select  a  mission  to  re-roll 
and  attempt  to  submit  it  to  the  system  only  to  have  the  database  constraint  checking 
mechanisms  tell  him  that  his  choice  is  ineligible  for  re-roll.  The  FIDO  might  have  to  do 
this  any  number  of  times  before  he  finds  a  valid  option  that  passes  the  system  checks. 
Therefore,  we  postulated  that  a  combination  of  fuselets  could  be  developed  to  derive  a 
list  of  valid  resource  options  to  attack  the  specified  target,  and  provide  other  supporting 
information  to  the  warfighter  to  help  the  FIDO  make  an  informed  decision  about  the 
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resource  he  should  select.  Fuselets  would  provide  a  proactive  decision-support 
capability,  as  opposed  to  a  reactive  one  where  there  is  a  high  potential  for  no-go  decisions 
where  replanning  has  to  back  up  and  repeat  itself. 


2.3.3  Develop  a  “To-Be”  Operational  Scenario  Process 

The  as-is  scenario  that  we  arrived  at  was  fairly  large,  and  our  time  and  budget  constraints 
relatively  limited.  Therefore,  we  chose  to  narrow  the  scope  of  our  experiment  to  a  small 
slice  of  the  as-is  process  for  technology  application.  We  decided  to  focus  on  supporting 
the  FIDO  for  the  decisions  he  has  to  make,  the  activities  he  has  to  perform,  and  the 
collaborations  he  has  with  other  duty  officers.  Figure  5  illustrates  the  as-is  slice  of  the 
process  that  we  defined.  Red  lines  indicate  potential  no-go  decisions  in  the  as-is  process 
that  our  technology  is  intended  to  help  minimize. 
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Figure  5  -  Subset  of  “ As-is ”  Dynamic  Mission  Replanning  Process 

Figure  6  depicts  our  to-be  operational  scenario  where  JBI,  JBI  fuselet,  and  collaborative 
fuselet  companion  clients  are  injected  into  the  overall  dynamic  mission  replanning 
scenario.  AOC  operators  have  access  to  all  of  the  same  legacy  C2  systems  and  tools  that 
they  are  accustomed  to  using,  as  well  as  the  capabilities  that  DISCOVER  affords  them. 
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Figure  6  -“To-Be”  Dynamic  Mission  Replanning  Process 


2.3.4  Requirements  Specification  and  Design  of  the  Solution 

Once  we  had  a  handle  on  the  operational  domain  to  which  we  were  to  apply  our 
technology,  we  had  to  define  the  requirements  for  each  of  the  system  components  within 
our  architecture.  This  involved  legacy  system  data  modeling  to  understand  the 
relationships  between  the  various  pieces  of  information  utilized  within  the  as-is  process, 
storyboarding  to  come  up  with  the  Graphical  User  Interfaces  (GUIs)  that  our 
collaboration  portal  client  would  present  to  the  end  user,  defining  use  cases  for  the 
various  interactions  between  the  users  and  the  system,  specifying  functional  descriptions 
of  the  system  components,  developing  sequence  diagrams  showing  the  flow  of  JBI 
information  object  publications,  subscriptions  and  queries,  and  finally  engineering  the 
information  object  schemas  for  the  objects  identified  in  the  sequence  diagrams.  Each  of 
these  activities  is  described  with  representative  examples. 

2.3.4. 1  Legacy  System  Data  Modeling 

We  test  drove  TBMCS  applications  and  studied  the  documentation  for  that  system  to 
derive  a  relational  data  model  using  Microsoft  Access.  This  data  model,  shown  in  Figure 
7,  proved  essential  in  better  understanding  the  as-is  process,  and  in  determining  what 
information  we  needed  to  extract  from  TBMCS  via  the  database  bridge  and  publish  to  the 
JBI  in  support  of  our  to-be  process.  It  also  allowed  us  to  prototype  how  fuselets  would 
correlate  various  information  objects  based  on  slices  through  the  data  model,  and  also 
helped  us  to  determine  what  additional  information  would  be  needed  to  augment  TBMCS 
in  the  to-be  process  (see  Figure  8). 
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Figure  7  -  Legacy  System  Data  Modeling 


Figure  8  -  Relational  Slice  through  the  Data  Model 
2.3.4.2  Storyboarding 

We  used  Microsoft  Visio  to  develop  some  notional  GUIs  for  the  collaboration  and 
visualization  capabilities  to  be  provided  by  the  Portal  Client  of  our  system.  Figure  9 
shows  one  of  these  screens. 
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Figure  9  -  Storyboarding  a  GUI  for  Messaging  and  Chat  Collaborations 

It  should  be  noted  that  the  actual  screens  developed  will  probably  look  somewhat 
different  from  those  that  we  have  storyboarded.  At  the  time  of  this  writing,  we  are  only 
two-thirds  of  the  way  through  the  project. 

2.3 .4.3  Use  Cases 

We  also  used  Microsoft  Visio  to  define  various  use  cases  tied  to  our  to-be  process. 

Figure  10  illustrates  some  of  these  Unified  Modeling  Language  (UML)  use  case 
diagrams.  We  were  able  to  capture  snippets  of  the  as-is  process  within  the  system  box  of 
each  use  case,  allowing  us  to  map  each  use  case  back  to  the  operational  scenario. 
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2.3. 4.4  Functional  Descriptions 

We  developed  functional  descriptions,  such  as  the  following  example:  “The  portal 
application  shall  allow  for  text  messaging  between  the  various  duty  officers  so  that  they 
can  collaborate  on  the  targeting  information  and  special  requirements  that  are  presented 
in  the  portal  application.  Note  that  users  shall  be  able  to  collaborate  via  asynchronous 
Email-like  alert  messages  as  well  as  via  synchronous  multi-party  chat  sessions.”4 

2.3.4.5  Sequence  Diagrams 

In  order  to  understand  how  to  coordinate  the  sequence  of  information  object  publications, 
subscriptions,  and  queries  by  the  UST  Client,  Database  Bridge,  Fuselets,  and  the  Portal 
Client,  we  developed  UML  sequence  diagrams  using  TogetherSoft  to  illustrate  the 
passing  of  information  objects  within  the  system,  all  of  which  occurs  transparently  to  the 
end  user.  Figure  1 1  illustrates  one  such  sequence  diagram. 
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Figure  11  -An  Example  of  a  UML  Sequence  Diagram 

2.3.4.6  Information  Engineering 

Once  we  had  identified  in  our  sequence  diagrams  the  information  objects  required  to 
support  our  system,  we  needed  to  specify  the  schemas  for  each  information  object  type. 
We  used  XMLSpy  for  XML  schema  development.  Figure  12  depicts  a  graphical 
representation  of  the  XML  Schema  for  one  of  our  information  object  types. 
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Figure  12  -  Graphical  View  of  an  Information  Object  Schema 

This  particular  example  is  the  output  of  one  fuselet  that  is  later  consumed  as  the  input  to 
another,  and  that  is  where  the  sequence  diagrams  become  an  important  tool  in 
understanding  workflow  dependencies  between  the  various  system  components. 

2.3.5  Portal  and  UST  Client  Design  and  Development 

We  hope  to  tie  together  all  the  collaboration  and  information  sharing  needs  of  AOC 
operators  within  a  flexible  and  extensible  portal  framework.  Ultimately,  in  our  view,  this 
portal  is  all  about  collaboration.  The  JBI  is  one  mechanism  for  sharing  specific  kinds  of 
information  between  users  while  collaboration  features  like  text  messaging  allow  other 
kinds  of  (usually  ad-hoc)  information  sharing.5  The  initial  DISCOVER  client  suite  that 
InfoDynamics  is  assembling  consists  of  these  primary  components: 

•  Jabber  Inc.  Messaging  Server  &  Web  Client 

•  Jakarta  Jetspeed  Portal  (v.  1.5) 

•  Tomcat  5.x 

•  Apache  1.3x  Web  Server 
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The  UST  Client  for  populating  the  JBI  information  space  with  military  unit  munition¬ 
handling  qualifications  is  a  simpler  client  to  develop  than  the  Portal  Client.  Figure  13 
shows  a  notional  look-and-feel  for  this  client. 
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Figure  13  -  Unit  Status  Technician  (UST)  Client 

2.3.6  SUKO  Fuselet  Design  &  Fuselet  Development 

Fuselets  that  maintain  information  about  their  internal  state  over  the  passage  of  time  are 
called  “stateful”  fuselets.  While  the  internal  state  of  a  fuselet  cannot  be  queried  directly 
by  other  JBI  clients,  the  fuselet  can  provide  a  “snap-shot”  of  this  information  by 
publishing  it  to  the  JBI  as  an  immutable  information  object.  A  stateful  fuselet  that 
maintains  state  information  based  on  inputs  received  from  multiple  information  sources 
(via  the  JBI)  and  publishes  this  information  periodically  according  to  some  business  logic 
for  consumption  by  multiple  clients  with  a  common  interest  in  (and  possible  contribution 
to)  that  information  is  what  is  known  as  a  “Shared  Updateable  Knowledge  Object” 
(SUKO).6 

In  DISCOVER,  we  came  up  with  an  initial  design  for  a  SUKO  fuselet  and  how  one 
would  be  triggered  to  publish  the  information  it  maintains.  In  addition  to  subscribing  for 
inputs  from  multiple  clients  for  information  transformation  according  to  some  business 
logic,  a  SUKO  fuselet  can  also  subscribe  for  a  “trigger”  information  object  that  is 
published  by  an  external  client  as  an  instruction  to  the  fuselet  when  to  publish  and  what 
to  publish.  Figure  14  illustrates  how  this  works. 
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Figure  14-  Triggering  a  SUKO  Fuselet 

What  is  depicted  in  Figure  14  is  a  SUKO  that  is  subscribing  to  information  objects  Si  and 
S2  that  are  published  to  the  JBI  by  Client  1  and  Client  2,  respectively.  Based  on  these 
inputs  and  the  manipulations  performed  on  them,  the  SUKO  internally  maintains  the 
result  Pi.  The  SUKO  also  subscribes  to  a  trigger  information  object  type  T.  In  this 
example,  Client  3  publishes  T  to  trigger  the  fuselet  to  publish  the  output  information 
object  P0  which  represents  the  current  state  of  P,.  The  JBI  then  delivers  P„  to  Client  3 
based  on  its  subscription  to  that  information  object.  The  SUKO  trigger  information  object 
that  we  designed  is  shown  in  Figure  15.  It  tells  the  SUKO  what  information  object(s)  it 
should  publish,  and  includes  optional  parameterization  that  can  be  used  to  prescribe  what 
exactly  the  fuselet  is  to  produce  as  information  in  its  publication. 


baseObject 
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Figure  15  -  Schema  of  a  SUKO  Trigger  Information  Object 
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In  DISCOVER,  the  SUKOs  we  are  developing  manipulate  inputs  from  each  of  the  major 
TBMCS  databases  (via  the  bridge),  the  UST  Client,  and  other  fuselets.  They  then  publish 
their  results  to  the  JBI  for  consumption  by  the  Portal  Client  when  a  SUKO  trigger 
information  object  is  received,  and  this  is  usually  associated  with  an  end  user  action  such 
as  a  button  click.  While  not  utilized  in  the  DISCOVER  project,  another  kind  of  SUKO 
maintains  trigger  information  internally  based  on  business  logic  that  dictates  when  and 
what  the  fuselet  should  publish  -  that  is,  with  this  kind  of  a  SUKO,  trigger  information 
objects  are  not  needed  to  invoke  fuselet  publications. 

Not  all  of  the  fuselets  being  developed  for  the  DISCOVER  project  are  SUKOs.  Some  are 
stateless  fuselets  that  perform  rather  simple  functions.  All  of  the  fuselets,  including 
SUKOs,  are  developed  using  the  Fuselet  Development  Environment  (FDE)  created  by 
General  Electric  and  ISX  Corporation  under  a  previous  contract  to  AFRL  in  support  of  a 
congressionally  funded  effort  called  Information  Management  for  Crisis  Response 
(IMCR),  another  JBI  experiment  focused  on  homeland  security.  The  FDE  provides  not 
only  an  Interactive  Development  Environment  (IDE)  for  authoring  fuselets,  but  also  a 
runtime  “sandbox”  for  testing,  debugging,  and  executing  fuselets.  AFRL  is  currently 
porting  this  sandbox  into  the  AFRL  JBI  fuselet  runtime  infrastructure  that  is  itself  under 
development  at  the  time  of  this  writing.7 

2.3.7  Controlled  Experimentation 

2.3.7. 1  Quantitative  and  Qualitative  Metrics 

The  DISCOVER  experiment  utilizes  the  Goal  Question  Metric  (GQM)  method  as  its 
measurement  system.8  The  GQM  method  was  originally  developed  by  V.  Basili  and  D. 
Weiss9,  and  expanded  with  many  other  concepts  by  D.  Rombach.10  We  are  using  the 
GQM  method  to  define  measurement  on  the  DISCOVER  project,  process,  and  products 
in  such  a  way  that  the  resulting  metrics  will  be  tailored  to  the  goals  of  dynamic  mission 
replanning  within  an  AOC.  GQM  defines  a  measurement  model  on  three  levels: 

•  Conceptual  level  (goal):  A  goal  is  defined. 

•  Operational  level  (question):  A  set  of  questions  is  used  to  characterize  the  assessment 
or  achievement  of  a  specific  goal. 

•  Quantitative  level  (metric):  A  set  of  metrics  is  associated  with  every  question  in  order 
to  answer  it  in  a  measurable  way. 

Some  of  the  goals  for  DISCOVER  are: 

•  Operators  spend  less  time  looking  for  information  and  more  time  on  warfighting  and 
decision  making. 

•  Operators  can  respond  more  quickly  with  reduced  effort,  fewer  errors  and  optimized 
solutions. 

•  AOC  operations  can  be  distributed  and  less  vulnerable  to  attack  than  are  centralized 
AOC  operations. 

As  the  experiment  progresses,  we  will  formulate  the  questions  and  metrics  associated 
with  these  and  other  project  goals  (see  Figure  16  for  more). 
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Goal 

Question 

Metric 

Mission  Replanning 

Improve  Mission  Replanning  Process 
Effectiveness 

Does  the  JBI  and  fuselets  lead  to  a  greater  number  of 
successfully  rerolled  mission? 

%  improvement  in  missions  rerolled  successfully 

Improve  Mission  Replanning  Process 

Efficiency 

Does  the  JBI  and  fuselets  lead  to  less  time  to  reroll  a  missions? 

%  reduction  in  time  to  reroll  a  complete  mission 

Improve  Mission  Replanning  Process  Value 

Does  the  JBI  and  fuselets  improve  mission  replanning  value 
[I.e  fewer  missons  to  reroll  faster]? 

%  improvement  in  missions  rerolled  successfully/%  reduction 
in  time  to  reroll  a  misison 

Decision-Making 

Improve  Decision-Making  Effectiveness 

Does  the  JBI  and  fuselets  make  mission  planners  successful  in 
choosing  qualified  missions  to  reroll? 

%  reduction  in  unqualified  missions  to  reroll 

Improve  Decision-Making  Efficiency 

Does  the  JBI  and  fuselets  help  mission  planners  choose 
qualified  missions  to  reroll  faster? 

%  reduction  in  time  to  select  a  qualified  mission  to  reroll 

Improve  Decision-Making  Value 

Can  mission  planner  make  select  qualified  missions  to  reroll- 
faster  with  the  JBI  and  fuselets? 

%  reduction  in  negative  missions  to  reroll/%  reduction  in  time 
to  select  a  qualified  mission  to  reroll 

Information  Value 

Improve  Information  Effectiveness 

Does  the  JBI  and  fuselets  produce  more  effective  information? 

If  you  agree  that  the  JBI  and  fuselets  produce  more  effective 
information  and  hence  improve  your  ability  to  reroll  mission  - 
by  what  percent  would  rate  the  improvement  in  information 
effectiveness? 

Improve  Information  Efficiency 

Does  the  information  JBI  and  fuselets  produce  lead  to  increased 
process  and  decision-making  effeciency? 

If  you  agree  that  the  information  JBI  and  fuselets  produce 
leads  to  decreases  the  time  you  need  to  reroll  mission  -  by 
what  percent  would  rate  the  improvement? 

Improve  Information  Value 

Does  the  JBI  and  fuselets  produce  higher  value  information? 

If  you  agree  that  the  information  JBI  and  fuselets  produce 
helps  you  make  more  effective  decision  faster  -  by  what 
percent  would  rate  the  improvement? 

Collaborative  Communications 

Improve  Communications  Effectiveness 

Does  the  JBI  and  fuselets  improve  communications 
effectiveness? 

(1)  %  increase  in  machine  to  machine  communications;  (2)  % 
reduction  in  person  to  person  communication;  (3)  Would  you 
say  that  the  JBI  and  fuselets  improve  you  abilitiy  to 
communicate  -  if  so  by  what  percent? 

Improve  Communications  Efficiency 

Does  the  JBI  and  fuselets  reduce  the  time  mission  planners 
spend  communicating? 

(1)  %  reduction  in  the  time  mission  planners  spend 
communicating; (2)  Would  you  say  that  the  JBI  and  fuselets 
the  time  you  spend  communicating  -  if  so  by  what  percent? 

Improve  Communications  Value 

Does  the  JBI  and  fuselets  improve  communications  value? 

If  you  agree  that  the  JBI  and  fuselets  improve  your  ability  to 
communicate  and  also  save  you  time  -  by  what  percent  would 
rate  the  improvement  in  communications  value? 

Figure  16-  DISCOVER  Goals,  Questions,  and  Metrics  (GQM) 

2.3.7. 2  Experiment  Execution  and  Measurement 

We  will  be  running  our  fuselets  and  portal  software  within  an  unclassified  AOC  setting 
here  at  AFRL  and  against  an  instrumented  JBI  platform.  The  instrumentation  capability 
was  developed  at  AFRL  as  a  general  purpose  JBI  status  and  health  monitoring  system.11 
Therefore,  we  should  be  able  to  gain  some  insight  into  the  runtime  characteristics  with 
regard  to  information  object  production  and  consumption  rates  (e.g.,  fuselet  efficiency). 

However,  the  most  important  measurements  will  be  with  respect  to  the  quantitative  and 
qualitative  metrics  related  to  DISCOVER’ s  goals  for  information  accuracy  and 
operational  value.  Measurement  will  be  based  on  the  metrics  that  we  derive  through  the 
application  of  the  GQM  method. 

2.3.1. 3  Experimentation  Data  Assessment  and  Reporting 

The  GQM  method  will  provide  us  with  a  measurement  plan  that  deals  with  the  particular 
set  of  problems  we  are  attempting  to  solve  and  the  set  of  rules  for  obtaining,  interpreting, 
assessing  and  reporting  data  from  the  experiment.  The  interpretation  should  give  us  the 
answers  we  are  looking  for  if  the  project  goals  are  attained.  Figure  17  provides  some  of 
the  assessment  metrics  associated  with  the  questions  that  are  related  to  project  goals. 
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Question 

TBMCS 

DISCOVER 

Assessment  Metric 

Mission  Replanning 

Does  the  JBI  and  fuselets  lead  to  a 
greater  number  of  successfully 
rerolled  mission? 

(1)  number  of  missions  rerolled;  (2) 
number  of  missions  rerolled 
successfully 

(1)  number  of  missions  rerolled; 

(2)  number  of  missions  rerolled 
successfully 

%  of  DISCOVER  missions  rerolled  successfully  - 
%  of  TBMCS  missions  rerolled  successfully 

Does  the  JBI  and  fuselets  lead  to  less 

time  to  reroll  a  missions? 

Mean  time  to  reroll  a  complete 
TBMCS  mission 

Mean  time  to  reroll  a  complete 
DISCOVER  mission 

[Mean  time  to  reroll  a  complete  TBMCS  mission  - 
Mean  time  to  reroll  a  complete  DISCOVER 
mission]  /  Mean  time  to  reroll  a  complete  TBMCS 
mission 

Does  the  JBI  and  fuselets  improve 
mission  replanning  value  [I.e  fewer 
missons  to  reroll  faster]? 

(1)  %  TBMCS  missions  rerolled 
successfully;  (2)  Mean  time  to  reroll 
a  TBMCS  mission 

(1)  %  DISCOVER  missions 
rerolled  successfully;  (2)  Mean 
time  to  reroll  a  DISCOVER 

mission 

[%  of  DISCOVER  missions  rerolled  successfully  - 
%  of  TBMCS  missions  rerolled  successfully]  / 
[Mean  time  to  reroll  a  complete  TBMCS  mission  - 
Mean  time  to  reroll  a  complete  DISCOVER 
mission]  /  Mean  time  to  reroll  a  complete  TBMCS 
mission 

Decision-Making 

Does  the  JBI  and  fuselets  make 
mission  planners  successful  in 
choosing  qualified  missions  to  reroll? 

(1)  number  of  TBMCS  missions 
selected  rerolled;  (2)  number  of 
unqualified  TBMCS  missions 
selected  to  rerolled 

(1)  number  of  DISCOVER 
missions  selected  rerolled;  (2) 
number  of  unqualified 

DISCOVER  missions  selected  to 

rerolled 

[number  of  unqualified  TBMCS  missions  selected 
to  rerolled  -  number  of  unqualified  DISCOVER 
missions  selected  to  rerolled]  /  [number  of 

TBMCS  missions  selected  reroll] 

Does  the  JBI  and  fuselets  help 
mission  planners  choose  qualified 
missions  to  reroll  faster? 

Mean  time  to  choose  a  qualified 
TBMCS  mission  to  reroll 

Mean  time  to  choose  a  qualified 
DISCOVER  mission  to  reroll 

[Mean  time  to  choose  a  qualified  TBMCS  mission 
to  reroll  -  Mean  time  to  choose  a  qualified 
DISCOVER  mission  to  reroll]  /  [Mean  time  to 
choose  a  qualified  TBMCS  mission  to  reroll] 

Can  mission  planner  select  more 
qualified  missions  to  reroll-faster  with 
the  JBI  and  fuselets? 

(1)  number  of  unqualified  TBMCS 
missions  to  reroll  selected  /  total 

number  of  TBMCS  missions  to 
reroll  selected;  (2)  Mean  time  to 
select  a  qualified  TBMCS  mission  to 
reroll 

(1)  number  of  unqualified 
DISCOVER  missions  to  reroll 

selected  /  total  number  of 

DISCOVER  missions  to  reroll 
selected;  (2)  Mean  time  to  select  a 
qualified  DISCOVER  mission  to 
reroll 

[number  of  unqualified  TBMCS  missions  to  reroll 
selected]  -  [number  of  unqualified  DISCOVER 
missions  to  reroll  selected]  /  [total  number  of 
TBMCS  missions  to  reroll  selected]  /  [Mean  time 
to  select  a  qualified  TBMCS  mission  to  reroll]  - 
[Mean  time  to  select  a  qualified  DISCOVER 
mission  to  reroll]  /  [Mean  time  to  select  a 
qualified  TBMCS  mission  to  reroll] 

Information  Value 

Does  the  JBI  and  fuselets  produce 
more  effective  information? 

What  percent  would  you  rate 
TBMCS's  information  effectiveness? 

What  percent  would  you  rate 
DISCO VER's  information 

effectiveness? 

[DISCOVER's  %  information  effectiveness]  -  [% 
TBMCS's  information  effectiveness]  /  [% 
TBMCS's  information  effectiveness] 

Does  the  information  JBI  and  fuselets 
produce  lead  to  increased  process  and 
decision-making  effeciency? 

What  percent  would  you  rate 
TBMCS's  process  and  decision¬ 
making  effeciency? 

What  percent  would  you  rate 
DISCO  VER's  process  and 
decision-making  effeciency? 

[DISCOVER's  process  and  decision-making 
effeciency]  -  [TBMCS's  process  and  decision¬ 
making  effeciency]  /  [TBMCS's  process  and 
decision-making  effeciency] 

Does  the  JBI  and  fuselets  produce 
higher  value  information? 

What  percent  would  you  rate 
TBMCS's  information  value? 

What  percent  would  you  rate 
DISCOVER's  information  value? 

[DISCOVER's  information  value]  -  [TBMCS's 
information  value]  /  [TBMCS's  information 
value] 

Collaborative  Communications 

Does  the  JBI  and  fuselets  improve 
communications  effectiveness? 

What  percent  would  you  rate 
TBMCS's  communications 

effectiveness? 

What  percent  would  you  rate 
DISCOVER's  communications 

effectiveness? 

[DISCOVER's  communications  effectiveness]  - 
[TBMCS's  communications  effectiveness]  / 
[TBMCS's  communications  effectiveness] 

Does  the  JBI  and  fuselets  reduce  the 
time  mission  planners  spend 
communicating? 

What  percent  of  the  mission 
replanning  time  do  TBMCS's 
planners  spend  communicationing? 

What  percent  of  the  mission 
replanning  time  do  DISCOVER's 
planners  spend 
communicationing? 

[%  TBMCS's  mission  replanning  time 
communicating]  -  [%  DISCOVER's  mission 
replanning  time  communicating]  /  [%  TBMCS's 
mission  replanning  time  communicating] 

Does  the  JBI  and  fuselets  improve 
communications  value? 

What  percent  would  you  rate 
TBMCS's  communications  value? 

What  percent  would  you  rate 
DISCOVER's  communications 

value? 

[DISCOVER's  communications  value]  - 
[TBMCS's  communications  value]  /  [TBMCS's 
communications  value] 

Figure  17  -  DISCOVER  Assessment  Metrics 

2.4  Results  and  Lessons  Learned 

While  the  DISCOVER  project  has  not  yet  completed  as  of  the  date  of  this  writing,  we 
have  already  learned  that  a  great  deal  of  application  domain  knowledge  needs  to  be 
assimilated  before  an  understanding  of  problems  and  the  development  of  potential 
solutions  can  be  embarked  upon.  It  seems  there  will  always  be  is  a  fairly  steep  learning 
curve,  either  for  the  fuselet  developer  to  understand  the  problem  domain,  or  for  the 
subject  matter  expert  to  learn  how  to  build  fuselets  and  companion  clients. 
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One  of  the  objectives  (not  the  primary  one)  of  the  DISCOVER  experiment  is  to  evaluate 
the  graphical  fuselet  authoring  capabilities  provided  by  the  GE/ISX  Fuselet  Development 
Environment  (FDE)  for  ease  of  use  by  people  that  are  not  proficient  programmers.  The 
intent  of  developing  the  FDE’s  graphical  fuselet  authoring  capabilities  was  to  empower 
non-programmer  domain  experts  with  the  ability  to  craft  fuselets  that  capture  the  business 
logic  for  which  they  have  the  best  understanding.  What  we  learn  in  this  regard  will  help 
shape  future  research  and  development  in  fuselet  production  environment  technology. 

The  primary  lessons  to  be  learned,  however,  have  yet  to  be  revealed  since  the  results  of 
the  experiment  are  not  yet  available.  It  is  the  expectation  at  this  time  that  results  and 
lessons  learned  will  be  ready  to  be  unveiled  in  time  for  the  10th  International  Command 
and  Control  Research  and  Technology  Symposium  (ICCRTS). 

3.  Conclusion 

In  this  paper,  we  have  presented  the  work  that  has  been  done  to  date  in  an  experiment 
designed  to  evaluate  and  validate  the  value  of  JBI  publish/subscribe  infrastructure,  JBI 
fuselet,  and  collaboration  technology  to  the  real-world  operational  domain  of  joint  air 
operations,  specifically  dynamic  mission  replanning.  We  hypothesize  that  the  technology 
can  make  positive  contributions  to  better  and  faster  decision  making  and  communication 
through  the  production  of  high-value,  decision-quality  information  while  doing  so 
unobtrusively  and  affordably.  Following  the  principles  of  the  scientific  method,  we  hope 
to  execute  a  controlled  experiment  to  validate  our  hypothesis,  and  be  able  to  present  the 
results. 
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